Quality of Large Language Model Responses to Radiation Oncology Patient Care Questions.

Importance: Artificial intelligence (AI) large language models (LLMs) demonstrate potential in simulating human-like dialogue. Their efficacy in accurate patient-clinician communication within radiation oncology has yet to be explored. Objective: To determine an LLM's quality of responses to radiation oncology patient care questions using both domain-specific expertise and domain-agnostic metrics. Design, Setting, and Participants: This cross-sectional study retrieved questions and answers from websites (accessed February 1 to March 20, 2023) affiliated with the National Cancer Institute and the Radiological Society of North America. These questions were used as queries for an AI LLM, ChatGPT version 3.5 (accessed February 20 to April 20, 2023), to prompt LLM-generated responses. Three radiation oncologists and 3 radiation physicists ranked the LLM-generated responses for relative factual correctness, relative completeness, and relative conciseness compared with online expert answers. Statistical analysis was performed from July to October 2023. Main Outcomes and Measures: The LLM's responses were ranked by experts using domain-specific metrics such as relative correctness, conciseness, completeness, and potential harm compared with online expert answers on a 5-point Likert scale. Domain-agnostic metrics encompassing cosine similarity scores, readability scores, word count, lexicon, and syllable counts were computed as independent quality checks for LLM-generated responses. Results: Of the 115 radiation oncology questions retrieved from 4 professional society websites, the LLM performed the same or better in 108 responses (94%) for relative correctness, 89 responses (77%) for completeness, and 105 responses (91%) for conciseness compared with expert answers. Only 2 LLM responses were ranked as having potential harm. The mean (SD) readability consensus score for expert answers was 10.63 (3.17) vs 13.64 (2.22) for LLM answers (P < .001), indicating 10th grade and college reading levels, respectively. The mean (SD) number of syllables was 327.35 (277.15) for expert vs 376.21 (107.89) for LLM answers (P = .07), the mean (SD) word count was 226.33 (191.92) for expert vs 246.26 (69.36) for LLM answers (P = .27), and the mean (SD) lexicon score was 200.15 (171.28) for expert vs 219.10 (61.59) for LLM answers (P = .24). Conclusions and Relevance: In this cross-sectional study, the LLM generated accurate, comprehensive, and concise responses with minimal risk of harm, using language similar to human experts but at a higher reading level. These findings suggest the LLM's potential, with some retraining, as a valuable resource for patient queries in radiation oncology and other medical fields.

Determining risk and predictors of head and neck cancer treatment-related lymphedema: A clinicopathologic and dosimetric data mining approach using interpretable machine learning and ensemble feature selection.

Background and purpose: The ability to determine the risk and predictors of lymphedema is vital in improving the quality of life for head and neck (HN) cancer patients. However, selecting robust features is challenging due to the multicollinearity and high dimensionality of radiotherapy (RT) data. This study aims to overcome these challenges using an ensemble feature selection technique with machine learning (ML). Materials and methods: Thirty organs-at-risk, including bilateral cervical lymph node levels, were contoured, and dose-volume data were extracted from 76 HN treatment plans. Clinicopathologic data was collected. Ensemble feature selection was used to reduce the number of features. Using the reduced features as input to ML and competing risk models, internal and external lymphedema prediction capability was evaluated with the ML models, and time to lymphedema event and risk stratification were estimated using the risk models. Results: Two ML models, XGBoost and random forest, exhibited robust prediction performance. They achieved average F1-scores and AUCs of 84 ± 3.3 % and 79 ± 11.9 % (external lymphedema), and 64 ± 12 % and 78 ± 7.9 % (internal lymphedema). Predictive ML and risk models identified common predictors, including bulky node involvement, high dose to various lymph node levels, and lymph nodes removed during surgery. At 180 days, removing 0-25, 26-50, and > 50 lymph nodes increased external lymphedema risk to 72.1 %, 95.6 %, and 57.7 % respectively (p = 0.01). Conclusion: Our approach, involving the reduction of HN RT data dimensionality, resulted in effective ML models for HN lymphedema prediction. Predictive dosimetric features emerged from both predictive and competing risk models. Consistency with clinicopathologic features from other studies supports our methodology.

GlioPredictor: a deep learning model for identification of high-risk adult IDH-mutant glioma towards adjuvant treatment planning.

Identification of isocitrate dehydrogenase (IDH)-mutant glioma patients at high risk of early progression is critical for radiotherapy treatment planning. Currently tools to stratify risk of early progression are lacking. We sought to identify a combination of molecular markers that could be used to identify patients who may have a greater need for adjuvant radiation therapy machine learning technology. 507 WHO Grade 2 and 3 glioma cases from The Cancer Genome Atlas, and 1309 cases from AACR GENIE v13.0 datasets were studied for genetic disparities between IDH1-wildtype and IDH1-mutant cohorts, and between different age groups. Genetic features such as mutations and copy number variations (CNVs) correlated with IDH1 mutation status were selected as potential inputs to train artificial neural networks (ANNs) to predict IDH1 mutation status. Grade 2 and 3 glioma cases from the Memorial Sloan Kettering dataset (n = 404) and Grade 3 glioma cases with subtotal resection (STR) from Northwestern University (NU) (n = 21) were used to further evaluate the best performing ANN model as independent datasets. IDH1 mutation is associated with decreased CNVs of EGFR (21% vs. 3%), CDKN2A (20% vs. 6%), PTEN (14% vs. 1.7%), and increased percentage of mutations for TP53 (15% vs. 63%), and ATRX (10% vs. 54%), which were all statistically significant (p < 0.001). Age > 40 was unable to identify high-risk IDH1-mutant with early progression. A glioma early progression risk prediction (GlioPredictor) score generated from the best performing ANN model (6/6/6/6/2/1) with 6 inputs, including CNVs of EGFR, PTEN and CDKN2A, mutation status of TP53 and ATRX, patient's age can predict IDH1 mutation status with over 90% accuracy. The GlioPredictor score identified a subgroup of high-risk IDH1-mutant in TCGA and NU datasets with early disease progression (p = 0.0019, 0.0238, respectively). The GlioPredictor that integrates age at diagnosis, CNVs of EGFR, CDKN2A, PTEN and mutation status of TP53, and ATRX can identify a small cohort of IDH-mutant with high risk of early progression. The current version of GlioPredictor mainly incorporated clinically often tested genetic biomarkers. Considering complexity of clinical and genetic features that correlate with glioma progression, future derivatives of GlioPredictor incorporating more inputs can be a potential supplement for adjuvant radiotherapy patient selection of IDH-mutant glioma patients.

Image-Based Deep Neural Network for Individualizing Radiotherapy Dose Is Transportable Across Health Systems.

PURPOSE: We developed a deep neural network that queries the lung computed tomography-derived feature space to identify radiation sensitivity parameters that can predict treatment failures and hence guide the individualization of radiotherapy dose. In this article, we examine the transportability of this model across health systems. METHODS: This multicenter cohort-based registry included 1,120 patients with cancer in the lung treated with stereotactic body radiotherapy. Pretherapy lung computed tomography images from the internal study cohort (n = 849) were input into a multitask deep neural network to generate an image fingerprint score that predicts time to local failure. Deep learning (DL) scores were input into a regression model to derive iGray, an individualized radiation dose estimate that projects a treatment failure probability of < 5% at 24 months. We validated our findings in an external, holdout cohort (n = 271). RESULTS: There were substantive differences in the baseline patient characteristics of the two study populations, permitting an assessment of model transportability. In the external cohort, radiation treatments in patients with high DL scores failed at a significantly higher rate with 3-year cumulative incidences of local failure of 28.5% (95% CI, 19.8 to 37.8) versus 10.2% (95% CI, 5.9 to 16.2; hazard ratio, 3.3 [95% CI, 1.74 to 6.49]; P < .001). A model that included DL score alone predicted treatment failures with a concordance index of 0.68 (95% CI, 0.59 to 0.77), which had a similar performance to a nested model derived from within the internal cohort (0.70 [0.64 to 0.75]). External cohort patients with iGray values that exceeded the delivered doses had proportionately higher rates of local failure (P < .001). CONCLUSION: Our results support the development and implementation of new DL-guided treatment guidance tools in the image-replete and highly standardized discipline of radiation oncology.

Deterministic small-scale undulations of image-based risk predictions from the deep learning of lung tumors in motion.

INTRODUCTION: Deep learning (DL) models that use medical images to predict clinical outcomes are poised for clinical translation. For tumors that reside in organs that move, however, the impact of motion (i.e., degenerated object appearance or blur) on DL model accuracy remains unclear. We examine the impact of tumor motion on an image-based DL framework that predicts local failure risk after lung stereotactic body radiotherapy (SBRT). METHODS: We input pre-therapy free breathing (FB) computed tomography (CT) images from 849 patients treated with lung SBRT into a multitask deep neural network to generate an image fingerprint signature (or DL score) that predicts time-to-event local failure outcomes. The network includes a convolutional neural network encoder for extracting imaging features and building a task-specific fingerprint, a decoder for estimating handcrafted radiomic features, and a task-specific network for generating image signature for radiotherapy outcome prediction. The impact of tumor motion on the DL scores was then examined for a holdout set of 468 images from 39 patients comprising: (1) FB CT, (2) four-dimensional (4D) CT, and (3) maximum-intensity projection (MIP) images. Tumor motion was estimated using a 3D vector of the maximum distance traveled, and its association with DL score variance was assessed by linear regression. FINDINGS: The variance and amplitude in 4D CT image-derived DL scores were associated with tumor motion (R2  = 0.48 and 0.46, respectively). Specifically, DL score variance was deterministic and represented by sinusoidal undulations in phase with the respiratory cycle. DL scores, but not tumor volumes, peaked near end-exhalation. The mean of the scores derived from 4D CT images and the score obtained from FB CT images were highly associated (Pearson r = 0.99). MIP-derived DL scores were significantly higher than 4D- or FB-derived risk scores (p < 0.0001). INTERPRETATION: An image-based DL risk score derived from a series of 4D CT images varies in a deterministic, sinusoidal trajectory in a phase with the respiratory cycle. These results indicate that DL models of tumors in motion can be robust to fluctuations in object appearance due to movement and can guide standardization processes in the clinical translation of DL models for patients with lung cancer.

Reducing the tracking drift of an uncontoured tumor for a portal-image-based dynamically adapted conformal radiotherapy treatment.

Accurate tracking of organ motion during treatment is needed to improve the efficacy of radiation therapy. This work investigates the feasibility of tracking an uncontoured target using the motion detected within a moving treatment aperture. Tracking was achieved with a weighted optical flow algorithm, and three different techniques for updating the reference image were evaluated. The accuracy and susceptibility of each approach to the accumulation of position errors were verified using a 3D-printed tumor (mounted on an actuator) and a virtual treatment aperture. Tumor motion up to 15.8 mm (peak-to-peak) taken from the breathing patterns of seven lung cancer patients was acquired using an amorphous silicon portal imager at ~ 7.5 frames/s. The first approach (INI) used the initial image detected, as a fixed reference, to determine the target motion for each new incoming image, and performed the best with the smallest errors. This method was also the most robust against the accumulation of position errors. Mean absolute errors of 0.16, 0.32, and 0.38 mm were obtained for the three methods, respectively. Although the errors are comparable to other tracking methods, the proposed method does not require prior knowledge of the tumor shape and does not need a tumor template or contour for tracking. Graphical abstract.

Application of TG-100 risk analysis methods to the acceptance testing and commissioning process of a Halcyon linear accelerator.

PURPOSE: A new type of linear accelerator (linac) was recently introduced into the market by a major manufacturer. Our institution is one of the early users of this preassembled and preconfigured dual-layer multileaf collimator (MLC), ring-gantry linac - Halcyon™ (1st version). We performed a set of full acceptance testing and commissioning (ATC) measurements for three Halcyon machines and compared the measured data with the standard beam model provided by the manufacturer. The ATC measurements were performed following the guidelines given in different AAPM protocols as well as guidelines provided by the manufacturer. The purpose of the present work was to perform a risk assessment of the ATC process for this new type of linac and investigate whether the results obtained from this analysis could potentially be used as a guideline for improving the design features of this type of linac. METHODS: AAPM's TG100 risk assessment methodology was applied to the ATC process. The acceptance testing process relied heavily on the use of a manufacturer-supplied phantom and the automated analysis of electronic portal imaging device (EPID) images. For the commissioning process, a conventional measurement setup and process (e.g., use of water tank for scanning) was largely used. ATC was performed using guidelines recommended in various AAPM protocols (e.g., TG-106, TG-51) as well as guidelines provided by the manufacturer. Six medical physicists were involved in this study. Process maps, process steps, and failure modes (FMs) were generated for the ATC procedures. Failure modes and effects analysis (FMEA) were performed following the guidelines given in AAPM TG-100 protocol. The top 5 and top 10 highest-ranked FMs were identified for the acceptance and commissioning procedures, respectively. Quality control measures were suggested to mitigate these FMs. RESULTS: A total of 38 steps and 88 FMs were identified for the entire ATC process. Fourteen steps and 34 FMs arose from acceptance testing. The top 5 FMs that were identified could potentially be mitigated by the manufacturer. For commissioning, a total of 24 steps and 54 potential FMs were identified. The use of separate measurement tools that are not machine-integrated has been identified as a cause for the higher number of steps and FMs generated from the conventional ATC approach. More than half of the quality control measures recommended for both acceptance and commissioning could potentially be incorporated by the manufacturer in the design of the Halcyon machine. CONCLUSION: This paper presents the results of FMEA and quality control measures to mitigate the FMs for the ATC process for Halcyon machine. Unique FMs that result from the differences in the ATC guidelines provided by the vendor and current conventional protocols, and the challenges of performing the ATC due to the new linac features and ring-gantry design were highlighted for the first time. The FMs identified in the present work along with the suggested quality control measures, could potentially be used to improve the design features of future ring-gantry type of linacs that are likely to be preassembled, preconfigured, and heavily reliant on automation and image guidance.